When an image or scene is captured on a camera or provided on some other electronic device or computer as a digital image, it can be desirable to modify the image in ways that require the device to segment foreground objects in the image from the background. For example, a user may want to change the background in the image for entertainment reasons, practical reasons such as to replace the background of a person speaking in a video conference to provide a background more appropriate or less distracting for business purposes, or artistic reasons. One such example of this is color pop where a foreground object is placed in color and the background is kept in black and white so that the foreground object is emphasized.
Also, it is sometimes desirable to modify the foreground in an image in real-time and on a preview screen of a camera for example as a scene is being recorded, such as to place emphasis on objects on a preview screen with the color pop by one example. This may be desirable for real-time applications such as with security and surveillance, video communication including tele-conferencing, computer vision, object recognition and augmentation, medical imaging, video coding efficiency, artistic or entertainment reasons, and others. Segmentation between background and foreground objects is used in object tracking to keep track of the objects from frame to frame in the video sequence during these real-time applications. Some precise segmentation algorithms are accurate but are often too slow for real-time applications such that a pause in the video can be noticeable to a user, and other segmentation algorithms are fast but can often be too inaccurate over time such that an error becomes more and more noticeable for each video frame analyzed by the faster segmentation algorithm.